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ABSTRACT 

The effects of college environments on student 
achievement on three GRE area achievement tests, when the initial 
ability of students was controlled were studied. A wide variety of 
environmental measures for 95 colleges were included. The method 
computed within-scnool regression lines, then colleges were clustered 
on regression lines, using a taxonomic technique, and multiple group 
discriminant functions using college descriptive variables were 
computed. It was found that in all three achievement areas one group 
or cluster of colleges could be identified as being more effective 
than the remaining groups of colleges. In the humanities areas the 
most effective group was characterized by a higher proportion of 
faculty with doctorates, greater selectivity, and a larger budget. In 
social sciences and the natural sciences, emphasis on subject matter 
area as measured by the proportion of majors in the appropriate 
fields of study was the only variable which demonstrated some 
discrimination between the most and least effective groups. (Author) 
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INTERACTION BETWEEN COLLEGE EFFECTS AND STUDENTS 1 APTITUDES 



Donald A. Rock, Leonard L. BaLrd, 
and Robert T . Linn 

Abstract 

The effects of college environments on student achievement on three 
GRE area achievement tests, when the initial ability of students was con- 
trolled, were studied. A wide variety of environmental measures for 95 
colleges were included. The method computed within-school regression lines, 
then colleges were clustered on regression lines, using a taxonomic tech- 
nique, and multiple group discriminant functions using college descriptive 
variables were computed. 

It was found that in all three achievement areas one group or cluster 
of colleges could be identified as being more effective than the remaining 
groups of colleges. In the humanities areas the most effective group was 
characterized by a higher proportion of faculty with doctorates, greater 
selectivity, and a larger budget. In social sciences and the natural 
sciences, emphasis on subject matter area as measured by the proportion of 
majors in the appropriate fields of study was the only variable which 
demonstrated some discrimination between the most and least effective groups. 



INTERACTION BETWEEN COLLEGE EFFECTS AND STUDENTS’ APTITUDES 



Donald A, Rock, Leonard L. Baird, 
and Robert L. Linn 

Most people assume that a student who goes to Harvard will learn more 
than a student who goes to Normal State. They assume that ’’better” pro- 
fessors, libraries, and ’’atmosphere” will result in more knowledge of 
science, humanities, and social science. But is it true? The average 
Harvard graduate may be more knowledgeable than the average Normal State 
graduate; however, the average Harvard student is also more knowledgeable 
than the average Normal State student when he enters college as a freshman. 
Thus, if Harvard enrolls only very bright students, and Normal State 
enrolls average students, Harvard seniors will naturally seem brighter than 
Normal State seniors at the end of college. The point is this: before 

we can tell whether Harvard or Normal State (or Swarthmore or Ohio State) 
is more effective, we have to control for the ability of the students 
going to those colleges. 

The idea of controlling for the ability of the students who attend 
various colleges seems simple and straightforward enough, but it involves 
many thorny statistical problems. If we do apply controls, how much of 
the outcome can we legitimately attribute to initial ability? How should 
we control for initial ability? Should we develop some score we would 
expect a student to get on the senior score based on his initial ability, 
then subtract this expected score from the actual score? Should the 
student or the college be the unit of analysis? Various analysis systems 
have been developed that offer different answers to these questions. 
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For example, a recent study of college effectiveness (Rock, Centra & 
Linn, 1970) indicated that there was a small but significant college 
effect on "achievement 11 when the school was the unit of analysis and the 
input was controlled. However, due to the methodological approach used 
only estimates of the overall effectiveness of a college across a broad 
range of student ability was tested. Feldman and Newcomb (1969) review 
more than 1,000 studies of the impact of college experience on student 
attitudes and achievement but almost all of them are only concerned with 
overall effectiveness. Astin (1970), in a much needed summarization of 
the methodology used in "college impact" studies, does discuss the concept 
of possible interaction between student aptitudes and college environment 
on achievement. He points out many of the pitfalls of applying standard 
statistical techniques to the interaction problem. 

Various systems for controlling for initial ability have been used but 
most seem to have considerable limitations. One of the most widely used 
models has been the "input-output" model. This model develops a predicted 
output score for students from their input scores, then subtracts this 
predicted score from the actual scores students obtain on the final measure. 
The average differences between expected and actual scores are then related 
to characteristics of the colleges. In general, the usual statistical 
controls used in the input-output model are based on logic underlying pre- 
diction models rather than explanatory models. As a result the inter- 
pretation of the part and/or partial correlation inherent in. the prediction 
models is often open to question (Linn & Werts, 1969). Also the various 
correlational techniques are not easily amenable to the identification of 
interactions or nonlinear effects. This study approaches the problem 
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in a way that appears to overcome some of the limitations of these 
methods. This approach is designed to find groups of colleges that are 
about equally effective for students with various levels of initial 
performance. Then the characteristics of the identified criterion groups 
are compared to see which characteristics are related to the relative 
effectiveness of the groups. For example* if colleges in more effective 
groups of colleges are smaller than colleges in less effective groups* we 
would assume that the size of a college is related to its effectiveness. 

But the advantage of this method is its definition of effectiveness. 

This method attempts to provide an intuitively simple approach which 
identifies both overall college effects and effects which interact with 
student ability. First, in order to come to any conclusions about the 
differential effects of the college we must treat the college as a sampling 
unit. However, the usual method of analyzing the resulting variation due 
to college mean output differences corrected for input ignores the within- 
college variation. In this study we made use of the withi.n-college varia- 
tion in estimating the interaction. Specifically, four steps were carried 
out: (1) all within-school regression lines were computed, i.e., Graduate 

Record Examination (GRE) Area tests were regressed on the College Entrance 
Examination Board Scholastic Aptitude Test (SAT) scores within school, 

(2) Ward’s (1963) hierarchical clustering technique was applied to group 
schools on the basis of the similarity of their regression lines, 

(3) multiple group discriminant functions using the regression parameters 
as the group discriminants were computed. This provided a test of whether 
the newly formed groups differed with respect to their pooled regression 
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lines, and (4) discriminant functions using college descriptive variables 
as the group discriminants were then computed. This method thus identified 
criterion clusters of colleges that differed in effectiveness by clustering 
on the slope, the mean SAT scores of the students, and the intercept. There- 
fore, one can identify and group colleges that have different effective- 
ness for students of different levels of initial ability. Then the. simul- 
taneous evaluation of the college descriptive variables which discriminate 
the clusters of colleges along with the relative slopes of their pooled 
within-group regression lines indicated the college characteristics which 
are associated with overall as well as differential effectiveness. 

Data and Analysis 

The data included a sample of 6,855 students from 95 colleges. The 
institutions in this study were largely private; only four were state colleges 
or universities. In general, student enrollment figures were modest; only 
10 colleges had more than 2,000 undergraduates, while 50 colleges enrolled 
fewer than 1,000. None of the colleges approached the large enrollments 
typified by some state and city universities. In addition to the public 
colleges and universities, the established, highly selective colleges in the 
Northeast were also underrepresented. Approximately half of the colleges in 
the sample were church related, with this group divided about equally between 
Catholic and Protestant denominations. In sum, the sample, while not 
representative of all American higher education, at least included the many 
types of small liberal arts institutions. 

Within each college a random sample of approximately 100 individuals 
was gathered, each of whom had SAT scores at their college entrance and the 



-5- 



GRL Area tests as upperclassmen. A total of 25 descriptive measures of the 
college were included in the analyses. These measures are listed in Table 1. 

Insert Table 1 about here 



Three separate taxonomies of the within-group regression lines were 
produced: one set was produced from the regressions of GRE Social Sciences 

on SAT-M; another set of groupings was produced from the regressions of 
GRE Humanities on SAT-v, and the last set was produced from the regressions 
of GRE Natural Science on SAT-M. In each case the one best predictor of 
any one criterion was used rather than both SAT-V and M, since in all 
three cases the increase in prediction was negligible. Also the stability 
of the college groupings based on the similarity of the within-college 
regression lines depends, of course, on holding to a minimum the number of 
parameters used to describe the regression lines. Thus the clustering of 
within-college regression lines was based on their similarity of intercepts, 
the regression weight and the mean predictor score for each college. Before 
clustering all three of these regression parameters were standardized. 

Ward’s (1963) grouping or clustering procedure as applied to the 
regression parameters starts with g groups and then forms g~l groups 
selectively collapsing groups which maximize the between- to the within- 
group sum of squares, continuing until the researcher notices a large incre- 
ment in the within-group sum of squares indicating the combining of two 
rather dissimilar groups and/or until the number of colleges within the 
smallest group becomes large enough to provide satisfactory future statis- 
tical estimates. The application of both of the above restraints led to 
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five groups for the humanities criterion, three for natural science, and 
four groups for social science. When the grouping was completed the students 
from colleges within a single group were pooled and the within-group regres- 
sion equation computed. 

Multiple group discriminant analyses were then performed to test for 
differences between the groups in each area on the college descriptive 
measures . 



Results 

Figure 1 shows the pooled within-group regression lines for the humanities 
area, the social sciences, and the natural sciences (the number of colleges 
in each group are shown in Table 3) . Inspection of the humanities regression 
plots indicates that the regression line for group 1 based on an N of 17 
colleges has a considerably steeper slope than groups 3 and 4 based on sample 
sizes 26 and 8 colleges respectively. The regression line for group 2 which 
was based on 13 colleges has a slope that was similar to the slope for group 
1, but a considerably different intercept. Although there appears to be some 
interaction between ability level and effectiveness for groups 2 and 3 the 
main effect associated with the group 1 colleges seems to be the overriding 
effect. Even of more interest here is the fact that groups 1, 2, and 5 have 
essentially the same input with respect to SAT-V yet their mean outputs differ 
by almost 100 points. There is a slight tendency for the colleges having 
lower input scores to also have flatter slopes, for example, group 3 and 
group 4 colleges. 

Insert Figure 1 about here 
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The social science regression plots were based on four groups. Similar 
to the humanities area there is one group of colleges based on an N of 9 
whose regression plot differs considerably from the remaining three groups. 
Unlike the humanities area, however, group 1 colleges differ primarily from 
the remaining groups of colleges only with respect to their intercept. On 
the basis of those plots we might suspect that group 1 colleges tend to 
emphasize achievement in those areas measured by the GRE Social Sciences. 

The natural sciences plots show three clusters that are similar with 
respect to input (SAT-M) but differ with respect to slope and intercept. 
However, as in both the humanities and the social sciences there is one 
cluster of colleges that is characterized by higher achievement. As in 
the humanities there is also one cluster of colleges, group 2, that not 
only has the lowest SAT-M mean but also has the flattest slope. It would 
seem that within the natural sciences and humanities the relationship between 
input and output is lowest for the schools characterized by lower inputs. 

At this point it seems reasonable to inquire whether the clusters of 
colleges are indeed significantly different with respect to their regression 
parameters. Thus discriminant functions were computed using the regression 
parameters associated with each college as the discriminants and their group 
membership as the dependent variable. In all three areas, humanities, social 
science, and natural science, at least two out of the three possible functions 
were significant (P < .01). In all cases the univariate F f s for intercept, 
slope, and SAT-V or M were highly significant (P < .01). Thus it may be 
concluded that the groups were internally homogeneous with respect to the 
regression parameters, yet significantly dif f eren t - f rom each other. 



O 

ERIC 



9 



Table 2 shows the results of the discriminant function analysis using 



Insert Table 2 about here 



college characteristics as discriminants for the five groups in the humanities 
area, the four groups in the social science, and the three groups in the 
natural science area. Although the canonical correlation for the natural 
science area was relatively high indicating fairly good discrimination it 
was not statistically significant, thus any further interpretation with re- 
spect to the size of the discriminant function coefficients was net warranted. 

Table 3 presents the five highest correlations between college character- 

insert Table 3 about here 



istics and the discriminant scores based on the first discriminant function 
from both humanities and social science areas. That is, the size of these 
correlations indicate their relative contribution to the particular function 
(in this case the first function) which best discriminates the groups in the 
humanities and social sciences. It is interesting to note that three of the 
five variables over lap ; that is, income per student, proportion of faculty with 
doctorate, and Astin’s (1965) measure of selectivity appear in both the 
humanities and the social science results. 

In order to gain insight into the relative ordering of the groups with 
respect to the variables which have the highest correlation with the discrimi- 
nant function score the group means and their associated univariate F tests 
were examined. Table 4 presents the group means and the univariate F's for 



-9- 



Insert Table 4 about here 



those variables having the highest correlation with the first discriminant 
function in the humanities and the social sciences. Since the overall or 
multivariate F for the natural sciences was not significant, further inter- 
pretations based on the univariate F’s for the natural sciences was not 
warranted . 

Humanities 

In Table 4 the patterns of the group means in the humanities area for 
the variables having the highest correlation with the discriminant scores 
reflect for the most part what one might expect from inspection of slopes 
in Figure 1. That is, groups 1 and 2 have for the most part the higher 
mean scores on the five significant college characteristics, especially 
selectivity, budget, and percentage of students graduating in four years. 
Group 1, of course, is the group of colleges whose pooled within-groups 
regression line in Figure 1 suggests a relatively large "main effect" with 
respect to achievement in the humanities. ""hat is, the predicted humanities 
achievement score given for a student entering with typical or average 
SAT-V scores for these groups of institutions would be expected to score 
more than 50 points higher than similar individuals attending any of 
the schools in the remaining four groups. This, of course, is a simple 
projection on the GRE axis from the point on group l ! s regression line 
defined by the SAT-V means in Figure 1. It is also interesting to note 
that with the exception of group 4 the remaining clusters of schools all 
have similar student inputs with respect to ability as measured by the SAT-V. 
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Group 4, of course, is characterized not only by considerably lower student 
ability input with respect to SAT-V but also has the lowest means on all 
five of the college characteristics. It should, however, come as no surprise 
that relatively low student input and low budgets seem to go hand in hand. 
Unfortunately these data do not allow us to infer what might be the effect 
on the slope and/or intercept if colleges with low ability inputs were in 
turn characterized by high budgets, proportion of faculty with doctorate, 
etc. For colleges characterized by similar and relatively higher verbal 
input, the humanities data do suggest that proportion of faculty with doctorate, 
size of budget, and selectivity are related to achievement. Selectivity is 
based on Astin’s (1965) work and is defined as the ratio of the number of 
national merit scholars who apply for admission to the total number of 
students in the entering freshman class. 

In order to examine the possibility that the high achieving schools in 
the humanities may have had a greater proportion of majors in the humanities 
area (reflecting a greater emphasis on course work in the humanities) , 
additional data were gathered and analyzed. It was found that on the average 
34% of the students in group 1 colleges major in the humanities as compared 
to 32% for the total sample. Although there is a tendency for group 1 
colleges to have a slightly larger proportion of students in humanities, it 
is doubtful whether an additional 2% of humanities majors can explain all 
of the over 50-point average increment in achievement associated with group 
1 colleges. 

Social Science 

In the social science area the slopes of all four clusters of colleges 
are quite similar yet the intercept for group 1 is considerably higher than 
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the remaining clusters, indicating that individuals attending these colleges 
can be expected to achieve, on the average, 60-70 points higher than in- 
dividuals of comparable SAT-M ability inputs who attended colleges in the 
other remaining three clusters. Although group 1 colleges appear to have 
this large main effect with respect to achievement in the social sciences, 
it is not reflected in the group means shown in Table 4. In fact, group 
_4 ! s means are the highest on all five variables. Group 4 is, of course, the 
least effective group of colleges with respect to social science achieve- 
ment according to the regression plots in Figure 1. Conversely, group 3, 
which is similar to group 4 with respect to its relative lack of effectiveness 
in the social sciences, has the lowest means on all five variables. Those 
inconsistencies in the pattern of group means relative to overall effective- 
ness in the social sciences area would lead one to conclude that additional 
data may be needed to explain the group ! s differential effectiveness. The 
fact that all the slopes are quite similar and the primary difference between 
groups lies in the intercepts suggests that differences between groups may be 
simply differential emphasis with respect to subject matter areas. 

Therefore, data on the proportions of students in the various subject 
areas were tabulated. Of particular interest was the comparison between 
groups 1 and 4 with respect to the average proportion of students majoring 
in social sciences. Surprisingly enough, the proportions were quite similar — 
19% in group 1, the most effective group, and 17.5% in group 4, the least 
effective group. Further examination of the data, however, revealed that 
group 1 had proportionally less than half as many students in the natural 
sciences (13.6%) as did group 4 (26.5%); group 1 also had proportionally 
twice as many students in the areas of education and business (30.6%) than 
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did group 4 (16.3%), the least effective group. Thus, it would seem that 
emphasis on subject content as measured by the proportion of students 
majoring in social science and allied areas may be the primary determinant 
of achievement in social science as measured by the GRE test. It would 
further seem that institutions which emphasize the natural science areas 
appear to accomplish this somewhat at the expense of the social science 
areas. The fact that group 4 institutions are characterized by both a 
high proportion of natural science majors and also the highest income per 
student also seems consistent with the possibility that education in the 
natural sciences may indeed require greater financial resources per unit of 
achievement than do the social sciences. In short, those schools that have 
limited income may have little choice but to place greater emphasis on the 
social science or related areas* 



1 Natural Sciences 

i 

i 

Since the canonical correlation was not significant, post hoc uni- 
variate F tests were not carried out. However, as in the humanities and 
social science areas, additional data were collected and analyzed with respect 
to proportion of majors in natural science. It was found that group 1, the 
most effective group of institutions in the natural sciences, had a slightly 
higher proportion of science majors (28%) than the average for the total 
sample (25%) . Group 1 also had a lower proportion of majors in the humanities 
(28%) than the total sample average of 32% c 

Discussion — _ 




These results suggest a tentative and complicated answer to our original 
question: Do students who attend one kind of college learn more than those 
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who attend another kind of college? The answer seems to be n yes, but...." 
First, even the significant differences between the colleges were generally 
small. A more heterogeneous sample may have resulted in larger differences. 
But even assuming sizable differences, it appears that the effective colleges 
were good at different things. That is, when we compared the most effective 
group of colleges in each area, there was little overlap in the colleges 
composing the groups. There was not one college that appeared in both the 
most effective group in humanities and the most effective group in social 
science. Only one college was in the most effective groups in humanities 
and natural science. In general, then, there did not seem to be any group 
of colleges that was among the best in every area. 

One possible explanation for the lack of significant results for the 
natural sciences items may stem from the content of ihe GRE area tests. 

Most of the items in the Natural Science area either require knowledge that 
could only be obtained in courses in science, or depend on mathematical 
reasoning that would presumably be present in students before college. In 
contrast, a sizable proportion of the items in the humanities and social 
science tests seem to require knowledge that could be learned outside the 
classroom, for example, that impressionism, is a school of art, while 
existentialism is a school of philosophy, or that members of the security 
council at the United Nations can exercise a veto. It is thus possible that 
knowledge of the humanities and social sciences may be related to general 
features of the environment, while knowledge of the natural sciences depends 
more on specific course work. This brings up the matter of distributional 
or course requirements. It is quite possible that the more effective colleges 
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in the sciences require more course work in the natural sciences regardless 
of the student ! s major field of study. If these data were available, it 
would be interesting to compare performances of colleges characterized by 
distributional requirements with those with more lax requirements. 

The humanities results suggest that students who go to selective 
colleges tend to achieve more than students who go to less selective 
colleges. From the data, it is hard to say whether this is due to the 
stimulating effect of other knowledgeable students or to some other charac- 
teristic of the colleges. 

Conclusions 

In comparing the most effective groups of colleges in each achievement 
area, it was found there was little overlap in the colleges composing the 
groups. Thus, in general, there was no one group of colleges which was 
"best" in every area. There was also a tendency for the less selective 
colleges in the humanities and natural science areas to be characterized 
by smaller relationships between their input and output. 

Finally, these results suggest some of the practical possibilities of 
i college impact research. The concern for symmetry in the information pro- 

i 

| vided to students and to institutions creates a need for a better under- 

| standing of the differential effects of colleges . Recently a study group 

e 

f commissioned by the College Entrance Examination Board made public their 

[ 

' recommendations which included providing evaluative information about 

colleges to students, i.e., having colleges take "tests." The purpose of 

such a college evaluation procedure would be to provide students with a 

I 

i 
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more accurate and detailed basis for choosing a college. Hopefully, the 
student could be presented with data which would indicate types or classes 
of colleges which would be particularly successful with his unique pattern 
of abilities and interests. Experience with the methodology used in this 
study suggests that certain colleges may be identified that are effective 
with students characterized by differing ability levels. That is, although 
a college may not demonstrate above average effects on achievement overall, 
it may yet show above average effects for low ability people or vice versa 
when considered separately. Thus, eventually we may have a data bank of 
information that could be used to guide and place students. The same infor- 
mation might be used by colleges for planning for the most effective use 
of their resources and for the allocation of their efforts. 
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Table 1 



College Descriptive Measures 
Used in Analyses 



Measures Derived from 
Office of Education Records 



Measures from 
Astin (1965) 



Number of Books in Library (Decile Ranking) 

Books per Student (Decile Ranking) 

College Income per Student (Decile Ranking) 

Number of Faculty per Student 
(Decile Ranking) 

Proportion of Faculty with Doctorate 
(Decile Ranking) 

Full Time Equivalent of Undergraduate 
Enrollment (Decile Ranking) 

College Expenditures per Student 

Type of College Control 

Percentage of Students Graduating in 
Four Years 

Percentage of Graduates Continuing to 
Graduate or Professional Schools 

Total Budget 

Percentage Freshman Male 



Intellectualism 

Estheticism 

S tatus 

Pragmatism 

Masculinity 

Estimated Selectivity 

Size 

Realistic Orientation 
Scientific Orientation 
Social Orientation 
Conventional Orientation 
Enterprising Orientation 
Artistic Orientation 
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Table 2 



First Function 
Second Function 



First Function 
Second Function 



First Function 



Discriminant Function Analysis Using College 
Characteristics as the Discriminants 



Humanities 



Canonical 


R 


Multivariate F 


Degrees of Freedom 


p 


.864 




1.428 


100, 149 


.024 


.668 




.843 


72, 113 


.668 






Social Sciences 






Canonical 


R 


Multivariate F 


Degrees of Freedom 


P 


.778 




1.397 


75, 144 


.053 


.689 




1.057 


4N 

00 

** 


.408 






Natural Sciences 






Canonical 


R 


Multivariate F 


Degrees of Freedom 


P 



.732 



1.273 

.877 



50, 78 
24, 40 



.167 



Second Function 



590 



627 
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Table. 3 



Correlations between the First Discriminant Function Score 
and Selected College Characteristics 

GRE AREAS b 



Humanities 



Income/Student. . . 14 

Proportion of Faculty 

with Doctorate. 13 

Selectivity 19 

Z Graduating in 

4 years 15 

Budget ... 13 



Social Sciences 



Income/Student. . . 20 

Proportion of Faculty 

with Doctorate. 26 

Selectivity 19 

Books 18 

% Going to Graduate 

School 20 



The five largest correlations with each function were selected. 



b 

Correlations for the Natural Sciences are not presented here since the 
Canonical R was not significant. 
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Table 4 

Group Means and Univariate F's for Selected College Characteristics 







Humanities 










College Characteristics 


Group 1 Group 2 Group 3 Group 

N = 17 N = 13 N = 26 N = 


4 Group 

8 N = 31 


5 


F 


P 


Income/Student 


6.06 


5.69 


6.15 5.00 


5.75 




.35 


.84 


Proportion of Faculty 
with Doctorate 


6.47 


5.69 


5.73 4.50 


5.34 




.97 


.43 


% Graduating in four years 


56.47 


60.22 


47.89 44.00 


55.59 




2.77 


.00 


Selectivity 


550.57 


523.62 


484.58 386.25 


514.71 




5.83 


.00 


Budget 


523.64 


493.62 


494.68 442.68 


484.28 




1.13 


.34 






Social 


Science 












G 1 


G 2 


G 3 G 4 


F 


P 








N = 9 


N - 42 


N = 33 N = 11 










Books 


5.56 


5.02 


4.03 6.80 


4.27 


.01 






Income/ Student 


5.85 


5.98 
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Fig. 1 

GROUP REGRESSION LINES BASED ON POOLED 
WITHIN SCHOOL REGRESSION LINES 
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